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Abstract 

Over  the  last  two  decades,  reducing  product  development  times  in  the  DoD  has  been 
the  focus  of  many  committees,  commissions,  and  research  efforts.  Despite  the 
implementation  of  numerous  recommendations,  the  DoD  still  struggles  with  long 
acquisition  cycle  times.  This  research  is  part  of  the  Air  Force  Cycle  Time  Reduction 
Research  Program  (CTRRP),  which  grew  out  of  the  Cycle  Time  Reduction  Action  Plan, 
developed  in  1998.  This  research  focuses  on  the  portfolio  management  (project  selection 
and  resource  allocation)  part  of  the  CTRRP. 

The  purpose  of  this  research  effort  was  to  investigate  of  the  use  of  portfolio 
management  within  the  Air  Force.  Specifically,  this  thesis  sought  to  assess  how  portfolio 
management  is  used  in  Air  Force  acquisition  and  to  compare  the  Air  Force’s  practices  to 
commercial  best  practices.  A  comprehensive  review  of  commercial  portfolio 
management  literature  was  conducted.  To  identify  Air  Force  practices,  semi-structured 
interviews  were  conducted  at  one  Air  Force  product  center.  Personnel  in  positions  most 
likely  to  use  portfolio  management,  or  have  knowledge  of  its  use,  were  interviewed  at  the 
center,  wing,  and  direct  reporting  group  levels. 

The  research  found  that  top  performing  commercial  firms  with  an  effective  portfolio 
management  process  focus  primarily  on  project  selection  activities  at  the  front  end  of  the 
development  process,  while  the  Air  Force  focuses  primarily  on  program  execution 
activities  at  the  back  end  of  the  process.  Recommendations  to  make  portfolio 
management  more  effective  in  the  Air  Force  are  discussed. 
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PROJECT  PORTFOLIO  MANAGEMENT: 


AN  INVESTIGATION  OF  ONE  AIR  FORCE  PRODUCT  CENTER 

I.  Motivation 

Long  Acquisition  Cycle  Times  in  the  DoD 

The  Department  of  Defense  (DoD)  is  one  of  the  largest  product  development 
organizations  in  the  world.  The  DoD  acquisition  budget  for  FY2005,  which  includes 
money  allocated  for  research,  development,  test,  evaluation,  and  procurement  of  new 
systems,  totaled  $143.8B.  The  Air  Force’s  portion  of  that  budget  was  $53. 7B,  more  than 
any  other  service  (DoD,  2004:1-2).  It  is  through  these  acquisition  activities  that  the  Air 
Force  equips  its  forces  with  the  tools  needed  to  accomplish  its  mission.  The  Air  Force 
must  have  an  effective  product  development  system  in  order  to  provide  the  right 
equipment  to  its  forces  at  the  right  time. 

Acquisition  cycle  time  is  a  critical  component  that  drives  product  development  times 

in  the  Air  Force.  Acquisition  cycle  time  is  defined  as  the  time  from  official  program 

initiation  to  initial  operating  capability  (DoD,  2001 : 1).  In  the  1960s,  the  average 

acquisition  cycle  time  in  the  DoD  was  84  months.  By  1996,  the  average  cycle  time  had 

grown  to  132  months  (DoD,  2001 :2).  In  2002,  the  Air  Force  Acquisition  Chief,  Dr. 

Marvin  Sambur,  addressed  his  concerns  with  long  cycle  times  when  he  stated: 

On  average,  Air  Force  programs'  cycle  times  run  about  10  years,  and  that's  only  the 
average;  some  programs  take  up  to  25  years  to  get  to  the  field.  When  it  takes  so  long, 
it  just  can’t  be  state-of-the-art.  (Paone,  2002) 
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Portfolio  management  is  a  tool  that  can  be  used  to  address  the  problem  of  long 
acquisition  cycle  times.  In  1998,  McNutt  conducted  research  into  the  role  of  the  schedule 
development  process  in  reducing  DoD  product  development  time.  McNutt  (1998)  found 
that  77%  of  projects’  schedules  were  limited  by  funding,  not  technology.  Lack  of 
adequate  funding  was  found  to  be  the  strongest  barrier  to  reducing  cycle  time.  The 
recommendations  of  McNutt  (1998)  are  summarized  in  the  Cycle  Time  Reduction  Action 
Plan  (CTRAP)  shown  in  Figure  1 .  This  research  will  build  upon  the  research  in  McNutt 
(1998)  by  focusing  on  the  “Project  Selection  and  Resource  Allocation”  part  of  Phase  3  of 
the  CTRAP. 


Figure  1  -  Cycle  Time  Reduction  Action  Plan  (McNutt,  2004) 
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Impact  of  Long  Acquisition  Cycle  Times 

The  DoD  has  long  recognized  that  increasing  acquisition  cycle  times  is  a  problem.  In 
1985,  the  President’s  Blue  Ribbon  Commission  on  Defense  Management  (also  known  as 
the  Packard  Commission)  was  established,  in  part,  to  evaluate  the  defense  acquisition 
system  and  determine  ways  to  acquire  products  with  lower  costs  and  shorter  development 
times  (Packard  Commission,  1986:1).  In  its  final  report,  the  Packard  Commission 
concluded: 

...a  much  more  serious  result  of  this  management  environment  is  an  unreasonably 
long  acquisition  cycle — ten  to  fifteen  years  for  our  major  weapon  systems.  This  is  a 
central  problem  from  which  most  other  acquisition  problems  stem.  (Packard 
Commission,  1986:8) 

The  commission  thought  it  would  be  possible  to  cut  DoD’s  acquisition  cycle  time  in  half 
by  streamlining  acquisition  organization  and  procedures,  using  technology  to  reduce 
costs,  balancing  cost  and  performance,  stabilizing  programs,  expanding  the  use  of 
commercial  products,  increasing  competition,  and  enhancing  the  quality  of  acquisition 
personnel  (Packard  Commission,  1986:15-27).  The  commission’s  recommendations 
quickly  became  law  but  did  little  to  correct  the  problem  because  the  goal  of  cutting 
development  time  was  not  internalized  by  the  DoD  (McNutt,  1998:52). 

The  DoD  continues  to  struggle  with  the  problem  of  long  cycle  times  today,  as 
evidenced  by  a  statement  in  2002  from  the  Director  of  the  U.S.  Office  of  Transformation, 
VADM  (Ret.)  Arthur  Cebrowski: 

If  program  managers  want  their  program  to  survive,  they  must  solve  and  resolve  the 
riddle  of  why  commercial  cycle  times  are  measured  in  weeks,  months  or  just  a  few 
years,  while  DoD’s  cycle  time  is  measured  in  decades.  (Griffard,  2002: 1) 
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Long  cycle  times  not  only  delay  the  delivery  of  the  proper  tools  to  our  warfighters,  they 
also  cause  many  other  problems.  As  the  Packard  Commission  stated  in  its  final  report, 
long  acquisition  cycle  times  are  the  cause  of  most  other  acquisition  problems. 

Systems  Not  Ready  When  Needed. 

Long  cycle  times  lead  to  systems  not  being  ready  when  needed.  McNutt  (1998) 
identified  seven  systems1  that  had  been  in  development  for  at  least  7  years  prior  to  the 
start  of  Desert  Storm.  Had  these  systems  been  ready,  they  would  have  satisfied  many 
critical  needs  in  the  early  part  of  the  war  (McNutt,  1998:40). 

Systems  Not  Meeting  Current  Needs. 

Long  cycle  times  lead  to  systems  not  meeting  current  needs  when  they  are  fielded. 
With  a  cycle  time  of  10-15  years,  the  current  military  and  political  enviromnent  will  have 
changed,  perhaps  drastically,  while  a  system  is  under  development.  Many  times  when  a 
system  is  finally  delivered,  the  requirements  for  that  system  have  changed  or  the  system 
is  no  longer  needed  (McNutt,  1998:41).  Air  combat  studies  conducted  by  the  Air  Force 
in  the  late  1970s  and  early  1980s  revealed  that  the  Soviet  Union  was  developing  two 
aircraft,  the  Fulcrum  and  the  Flanker  that  could  outperfonn  the  F-15.  In  1981,  the  Air 
Force  developed  a  requirement  for  an  Advanced  Tactical  Fighter,  which  would  become 
the  F-22.  Eight  years  later,  the  Soviet  Union,  which  was  the  primary  threat  the  F-22  was 
designed  to  counter,  collapsed  with  the  fall  of  the  Berlin  Wall.  In  an  attempt  to  have  the 
F-22  meet  more  current  needs  and  make  it  more  cost  effective,  the  Air  Force  changed  the 


1  The  seven  systems  were  the  C-17,  MILSTAR  communications  satellite,  LANTIRN  Precision  Targeting 
Pod,  Joint  Tactical  Information  Distribution  System,  Global  Positioning  System,  Advanced  Medium  Range 
Air-to-Air  Missile,  and  Sensor-Fuzed  Weapon  wide-area  anti-tank  capability  (McNutt,  1998). 
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designation  of  the  F-22  to  F/A-22.  There  is  also  an  F/B-22  bomber  concept  on  the  shelf 
for  future  consideration  (F/A-22,  2003). 

Systems  Fielded  with  Outdated  Technology. 

Long  cycle  times  lead  to  systems  being  fielded  with  outdated  technology.  Often 
times,  technology  that  is  current  at  the  start  of  a  program  is  obsolete  or  out  of  production 
by  the  time  the  system  is  fielded  (McNutt,  1998:41).  For  example,  computer  processing 
technology  is  advancing  very  rapidly.  According  to  Moore’s  Law,  the  number  of 
transistors  per  integrated  circuit  doubles  every  2-4  years  (Moore,  1965).  This  doubling 
results  in  dramatic  increases  in  the  number  of  instructions  per  second  that  a  processor  can 
execute  (see  Figure  2).  Imagine  fielding  a  system  today  that  began  development  15  years 
ago,  based  on  486-processor  technology.  Today’s  home  computers  have  processors  that 
are  on  the  order  of  one  thousand  times  faster.  If  the  Air  Force  ever  wants  to  field  a 
system  with  current  technology,  then  it  must  reduce  acquisition  cycle  times. 
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Figure  2  -  Computer  processor  performance  in  millions  of  instructions  per  second  (Moore,  2003) 
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Slow  Response  to  Emerging  Threats. 

Long  cycle  times  reduce  our  ability  to  respond  to  new  and  emerging  threats.  This 
leaves  our  forces  vulnerable  while  we  are  developing  a  new  system  to  counter  the  threat. 
One  example  is  the  AIM-9X  off-boresight  missile.  The  Soviet  Union  fielded  an  off- 
boresight  missile  on  their  aircraft  in  1985  (McNutt,  1998:43).  Eighteen  years  after 
identifying  the  threat,  the  U.S.  finally  deployed  a  counter-system  when  the  AIM-9X 
reached  initial  operating  capability  (IOC)  in  2003  (Navy,  2003). 

Increased  Costs. 

Long  cycle  times  cost  more  than  short  cycle  times.  In  its  report,  the  Packard 
Commission  stated,  “time  is  money,  and  experience  argues  that  a  ten-year  acquisition 
cycle  is  clearly  more  expensive  than  a  five-year  cycle”  (Packard  Commission,  1986:8). 
Programs  with  higher  development  costs  will  then  have  less  money  available  for 
production,  which  translates  to  fewer  systems  in  warfighters’  hands  (McNutt,  1998:45). 
Longer  cycle  times  also  increase  sustainment  costs  by  delaying  the  replacement  of 
systems  that  have  higher  operating  costs  (McNutt,  1998:49). 

Increased  Program  Instability. 

Long  cycle  times  increase  the  uncertainty  in  the  planning  process.  According  to 
McNutt  (1998),  programs  taking  less  than  7  years  to  reach  IOC  exceed  their  initial  budget 
estimates  by  an  average  of  15%,  compared  to  programs  taking  more  than  14  years  to 
reach  IOC  that  exceed  initial  estimates  by  an  average  of  42%  (McNutt,  1998:47).  Also, 
with  military  members  changing  positions  on  average  every  3-4  years,  long  cycle  times 
lead  to  a  high  rate  of  management  turnover.  This  high  turnover  increases  the  instability 
in  the  program. 
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Previous  Efforts  to  Reduce  Cycle  Times 

As  previously  stated,  the  DoD  has  long  recognized  the  importance  of  reducing 
acquisition  cycle  times.  This  section  summarizes  several  cycle  time  reduction  efforts 
conducted  within  the  Air  Force  over  the  last  10  years. 

Air  Force  Lightning  Bolt  #10  Initiative. 

The  Air  Force  Lightning  Bolt  #10  Initiative  originally  proposed  to  cut  in  half  the  time 
from  initial  effort  to  satisfy  a  requirement  until  delivery  of  the  product  or  service  (Air 
Force,  1996).  However,  once  underway,  the  initiative’s  scope  was  quickly  changed  to 
reducing  the  time  from  initial  effort  until  contract  award  (McNutt,  1998:53).  The 
Lighting  Bolt  group  developed  a  toolbox  of  best  practices  and  then  disbanded  (Air  Force, 
1997). 

Advanced  Concept  Technology  Demonstrations  (ACTDs). 

ACTDs  are  a  way  for  users  to  evaluate  a  new  technology’s  military  utility  without 
committing  to  a  development  effort.  ACTDs  do  not  fall  within  the  formal  acquisition 
process  and  have  much  more  flexibility  with  their  contracts.  Once  accepted,  ACTDs  are 
allowed  to  proceed  immediately  to  low-rate  initial  production.  While  ACTDs  effectively 
get  new  technology  into  warfighters’  hands  quickly,  they  do  not  solve  the  problem  of 
long  development  times.  They  are  simply  a  way  to  circumvent  the  traditional 
development  process  (McNutt,  1998:54-55). 

Lean  Aerospace  Initiative. 

The  Lean  Aerospace  Initiative  is  an  ongoing  effort  “to  reduce  cost,  development,  and 
production  time  for  military  products  by  half  by  infusing  commercial  lean  practices 
throughout  the  defense  aerospace  industry”  (McNutt,  1998:55).  The  initiative,  led  by  the 
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Massachusetts  Institute  of  Technology,  consists  of  a  consortium  of  government,  industry, 
academia,  and  labor  participants  conducting  research  into  the  development  and 
manufacturing  process  (McNutt,  1998:55). 

Warfighter  Rapid  Acquisition  Process  (WRAP). 

Air  Force  leadership  directed  implementation  of  the  WRAP  at  the  CORONA 
conference  in  November  1999.  The  process  was  designed  as  a  way  to  quickly  initiate  and 
fund  a  limited  number  of  high  value  projects  each  year  that  are  not  within  the  scope  of  an 
existing  program.  Major  commands  (MAJCOMs)  are  allowed  to  nominate  programs  for 
WRAP  funding,  but  must  agree  to  fully  fund  the  follow-on  program.  WRAP  allows  rapid 
transition  and  fielding  of  the  most  successful  warfighter  experiments,  Battlelab 
initiatives,  science  and  technology  efforts,  ACTDs,  and  other  novel  ideas  (McNutt, 
2000b). 

Cycle  Time  Reduction  Action  Plan  (CTRAP). 

The  Air  Force  Cycle  Time  Reduction  Team  was  chartered  in  1997  by  the  Assistant 
Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ)  and  the  Air  Force  Deputy  Chief  of 
Staff  for  Installations  and  Logistics  (AF/IL).  By  May  1998,  the  team  had  developed  a 
comprehensive  action  plan,  laying  out  the  steps  necessary  to  achieve  dramatic  reductions 
in  development  times  for  Air  Force  and  DoD  products.  Their  plan  was  approved  by  the 
Vice  Chief  of  Staff  in  August  1999.  Figure  1  presented  above  shows  a  summary  of  the 
CTRAP.  The  plan  has  three  phases.  Phase  one  includes  building  awareness  of  the 
problem  and  quantifying  its  impacts.  Phase  two  includes  building  the  necessary 
infrastructure  and  tools  necessary  to  fix  the  problem.  Phase  three  involves  mitigating 
funding  based  constraints  to  fixing  the  problem.  The  CTRAP  has  now  evolved  into  the 
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Cycle  Time  Reduction  Research  Program,  of  which  this  research  effort  is  a  part  (McNutt, 
2000a). 

Agile  Acquisition. 

In  March  2002,  Secretary  of  the  Air  Force  James  Roche  and  Air  Force  Chief  of  Staff 
General  John  Jumper  issued  a  mandate  to  the  Air  Force  acquisition  community  that  they 
must  change  the  way  they  do  business  in  order  to  more  rapidly  deliver  capability  to  the 
warfighter.  In  a  policy  memo  dated  4  February  2003,  Air  Force  Acquisition  Chief  Dr. 
Marvin  Sambur  implemented  Agile  Acquisition  with  two  overarching  objectives: 
decrease  acquisition  cycle  time  and  increase  credibility  in  execution.  Dr.  Sambur  stated 
that  his  goal  was  to  reduce  our  average  acquisition  cycle  time  by  a  factor  of  4: 1 .  To 
achieve  this  goal,  the  Air  Force  adopted  evolutionary  acquisition  and  spiral  development 
as  a  standard  practice.  Evolutionary  acquisition  is  a  process  designed  to  deliver 
capabilities  to  the  warfighter  faster  than  a  traditional  single-step  acquisition  approach. 
Spiral  development  is  another  process,  typically  used  in  conjunction  with  evolutionary 
acquisition,  which  places  a  strong  emphasis  on  program  risk  reduction.  Both  processes 
were  first  used  in  DoD  software  acquisition.  In  order  to  build  credibility  in  program 
execution,  the  Air  Force  focused  on  three  concepts:  collaborative  requirements — the 
warfighter,  acquirer,  and  tester  working  together  throughout  the  requirements  generation 
and  development  process;  seamless  verification — merging  developmental  and  operational 
testing  into  one  seamless  verification  process;  and  focused  technology — focusing  limited 
science  and  technology  funding  on  programs  that  directly  support  a  warfighting 
capability  and  bringing  technology  into  more  mature  programs  (Sambur,  2003). 
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Portfolio  Management:  A  Cycle  Time  Reduction  Tool 

A  project  portfolio  is  a  list  of  a  company’s  active  projects.  One  group  of  experts 

defines  portfolio  management  in  the  following  manner: 

Portfolio  management  is  a  dynamic  decision  process,  whereby  a  business’s  list  of 
active  new  product  (and  R&D)  projects  is  constantly  updated  and  revised.  In  this 
process,  new  projects  are  evaluated,  selected,  and  prioritized;  existing  projects  may  be 
accelerated,  killed,  or  deprioritized;  and  resources  are  allocated  and  reallocated  to  the 
active  projects.  The  portfolio  decision  process  encompasses  or  overlaps  a  number  of 
decision  making  processes  within  the  business,  including  periodic  reviews  of  the  total 
portfolio  of  all  projects  (looking  at  the  entire  set  of  projects,  and  comparing  all 
projects  against  each  other);  making  Go/Kill  decisions  on  individual  projects  on  an 
ongoing  basis;  and  developing  a  new  product  strategy  for  the  business,  complete  with 
strategic  resource  allocation  decisions.  (Cooper  et  al. ,  1998:3) 

Managing  a  project  portfolio  is  similar  to  managing  a  financial  portfolio.  Portfolio 

management  is  a  decision-making  process  used  to  balance  risk  and  reward  in  order  to 

maximize  an  organization’s  return  on  investment. 

Over  the  last  35  years,  the  average  acquisition  cycle  time  for  an  Air  Force  program  has 

grown  from  just  under  60  months  to  over  120  months  (McNutt,  2004).  During  the  same 

period,  commercial  industries  have  made  dramatic  reductions  in  their  product 

development  times  (McNutt,  1998:60).  Figure  3  shows  examples  of  some  of  these 

reductions.  Portfolio  management  is  one  of  the  key  tools  companies  have  used  to 

accomplish  these  reductions.  Researchers  have  studied  portfolio  management  in  the 

commercial  world  for  many  years.  However,  very  little  research  has  been  done  into  the 

application  of  portfolio  management  to  government  new  product  development. 

Recognized  or  not,  there  is  a  strong  need  for  this  type  of  research  in  the  DoD.  Figure  4 

shows  how  acquisition  cycle  times  have  grown  in  the  DoD  since  1969. 
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Figure  3  -  Commercial  reductions  in  cycle  times  (McNutt,  1998:66) 


Figure  4  -  Average  DoD  acquisition  cycle  times  (McNutt,  2004) 
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Research  Objectives 

The  objective  of  this  research  was  to  assess  the  use  of  portfolio  management  in  the  Air 
Force  acquisition  process.  This  was  accomplished  through  an  in-depth,  exploratory 
investigation  of  one  Air  Force  product  center.  Semi-structured  interviews  were 
conducted  with  personnel  in  key  positions  at  the  center,  wing,  and  direct  reporting  group 
(DRG)  levels.  Data  from  the  interviews  was  used  to  answer  several  questions:  1)  Does 
the  Air  Force  use  portfolio  management?,  2)  If  so,  how  do  they  use  it?,  3)  How  do  the  Air 
Force’s  activities  compare  to  commercial  portfolio  management  best  practices?,  and  4) 
What  should  the  Air  Force  do  to  improve  the  effectiveness  of  its  portfolio  management 
activities?  This  research  does  have  some  limitations.  It  does  not  address  the  use  of 
portfolio  management  by  other  product  centers,  other  acquisition  organizations,  or  the 
corporate  level  Air  Force.  The  research  also  does  not  address  modernization  programs 
within  the  major  commands  (MAJCOMs). 
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II.  Literature  Review 


One  of  the  difficulties  in  portfolio  management  is  that  the  concept  means  something 
different  to  everyone  in  an  organization.  Some  see  portfolio  management  as  building  a 
strategically  correct  portfolio — one  where  all  of  the  projects  directly  support  the 
organization’s  strategy.  Some  think  portfolio  management  is  about  the  most  effective 
way  to  allocate  resources.  Others  focus  on  building  a  portfolio  of  projects  that  will  lead 
to  future  breakthroughs  in  technology.  Still  others  look  to  portfolio  management  as  a 
way  to  ensure  the  organization  only  starts  projects  that  will  provide  immediate  financial 
gains.  In  reality,  portfolio  management  encompasses  all  of  these  goals. 

Research  into  Commercial  Portfolio  Management  Activities 

According  to  Poolton  and  Barclay  (1998),  many  researchers  have  investigated  the 
question  of  how  best  to  manage  a  portfolio  of  projects.  They  have  discovered  that  there 
is  more  than  one  answer  to  that  question.  After  nearly  five  decades  of  research,  there  still 
is  not  a  single  one  size  fits  all  model  that  explains  new  product  development  success. 
Prior  to  the  1970s,  most  research  focused  on  identifying  factors  associated  with 
successful  projects  and  identifying  factors  associated  with  failed  projects.  Since  then, 
research  has  shifted  to  comparing  successful  endeavors  with  those  that  are  unsuccessful. 
Researchers  have  taken  a  generic  approach  to  the  problem,  trying  to  identify  best 
practices  that  should  be  adopted  by  every  business,  regardless  of  the  size  or  complexity 
of  its  new  product  development  (Poolton  and  Barclay,  1998:199). 

Griffin  conducted  a  survey  in  1997  to  update  the  Product  Development  and 
Management  Association’s  1990  survey  of  commercial  new  product  development  best 
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practices.  The  survey  results  indicated  that  process  was  the  strongest  differentiator 
between  top  performing  firms  and  the  rest.  The  most  successful  companies  operated 
under  a  formal  new  product  development  process  based  on  the  organization’s  overall 
strategy.  They  were  also  more  likely  to  have  used  their  process  longer  than  lower 
performing  firms.  Finally,  the  top  performing  companies  were  found  to  focus  on 
improving  perfonnance  in  all  aspects  of  their  new  product  development  as  opposed  to 
just  one  aspect  (Griffin,  1997). 

Scott  (2000)  approached  his  research  a  bit  differently.  He  used  a  three-questionnaire 
DELPHI  methodology  to  rank  the  top  24  management  issues  involved  in  high-tech  new 
product  development.  The  primary  issue  identified  by  the  academia  and  industry 
participants  was  “strategic  planning  for  technology  products.”  Most  participants  agreed 
that  companies  have  a  difficult  time  implementing  and  following  through  with  a  long¬ 
term  technology  strategy.  Even  when  firms  do  have  a  well-structured  plan  in  place,  they 
often  abandon  it  for  short-tenn  opportunities  that  arise.  Other  issues  ranked  in  the  top 
five  were  new  project  selection,  organizational  learning,  core  competencies,  and  cycle 
time  reduction  (Scott,  2000). 

More  recently,  Jeffery  and  Leliveld  (2004)  conducted  a  survey  of  130  Fortune  1000 
companies.  The  survey  was  designed  to  1)  measure  the  adoption  of  infonnation 
technology  portfolio  management  (ITPM),  2)  identify  ITPM  implementation  hurdles,  3) 
assess  the  benefits  of  ITPM,  4)  define  ITPM  best  practices,  and  5)  build  strategies  for 
success.  The  major  result  of  their  efforts  was  the  IT  Portfolio  Management  Maturity 
Model  (see  Figure  5),  which  is  used  to  assess  ITPM  best  practices.  Jeffery  and  Leliveld’s 
(2004)  findings  indicate  that  only  17%  of  the  companies  surveyed  are  operating  at  the 
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synchronized  level,  despite  the  fact  that  65%  of  chief  information  officers  (CIOs)  believe 
that  adopting  ITPM  would  yield  significant  value  (Jeffery  and  Leliveld,  2004). 
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Figure  5  -  The  IT  Portfolio  Management  Maturity  Model  (Melymuka,  2004) 
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Portfolio  Management  Goals  and  Tools. 

By  far,  the  most  extensive  research  into  new  product  development  and  portfolio 
management  has  been  conducted  by  Cooper,  Edgett,  and  Kleinschmidt.  Over  several 
decades,  they  surveyed  hundreds  of  corporations  and  conducted  hundreds  of  case  studies 
on  new  product  developments.  They  discovered  that  the  most  successful  firms  do  not 
rely  on  a  single  approach  to  portfolio  management.  In  their  book  Portfolio  Management 
for  New  Products,  Cooper  et  al.  (1998)  give  a  very  comprehensive  definition  of  portfolio 
management  by  discussing  three  goals  of  portfolio  management  and  the  tools  that  support 
them.  The  three  goals  are  maximizing  value,  achieving  balance,  and  establishing  a  strong 
link  to  strategy. 

Maximizing  Value. 

According  to  Cooper  et  al.  (1998),  the  first  goal  of  many  firms  is  to  maximize  the 
value  of  the  projects  in  their  portfolio  in  tenns  of  a  given  business  objective,  such  as 
profitability,  return  on  investment,  likelihood  of  success,  or  some  other  strategic 
objective.  There  are  a  number  of  tools  (financial  methods,  scoring  models,  etc.)  used  to 
support  this  goal.  The  result  of  each  tool  is  a  prioritized  list  of  projects,  with  projects  at 
the  top  of  the  list  ranking  highest  in  terms  of  the  specific  objective.  The  challenge  for  the 
business  is  to  determine  the  appropriate  tool  and  criteria  with  which  to  rank  their  projects 
(Cooper  et  al.,  1998:  23-54). 

Achieving  Balance. 

The  second  goal  of  many  firms  is  to  achieve  a  balanced  portfolio.  Balance  can  be 
defined  in  terms  of  a  number  of  parameters:  long-term  versus  short-term,  high-risk  versus 
low-risk,  different  markets  the  business  is  involved  in,  different  technologies,  and 
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different  project  types.  Visual  charts  are  the  most  popular  way  of  displaying  project 
balance.  These  charts  include  portfolio  maps  (or  bubble  diagrams)  that  map  projects  on 
risk  versus  reward,  ease  of  implementation  versus  market  attractiveness,  ease  versus 
importance,  and  market  risk  versus  technology  risk.  Other  types  of  charts  used  are 
traditional  histograms,  bar  charts  and  pie  charts  (Cooper  et  al,  1998:  55-81). 

Strong  Link  to  Strategy. 

Regardless  of  all  other  considerations,  a  firm’s  final  portfolio  of  projects  must  match 
its  overall  strategy.  Firms  implement  strategy  by  spending  money;  therefore,  strategy  and 
resource  allocation  are  intimately  connected.  The  companies  studied  by  Cooper  et  al. 
(1998)  used  three  approaches  to  ensure  strategic  alignment:  top-down,  bottom-up,  and  a 
combination  of  both.  The  top-down  approach  is  also  called  the  strategic  buckets  model. 
Management  decides  the  strategy  for  the  firm,  determines  where  to  spend  the  firm’s 
resources,  and  then  allocates  the  appropriate  amount  of  funds  into  buckets.  Projects  are 
then  prioritized  within  the  buckets.  In  the  bottom-up  approach,  the  emphasis  is  placed  on 
project  selection.  Strategic  criteria  are  built  into  the  selection  process  so  that  the  resulting 
portfolio  of  projects  will  be  on  strategy  and  strategically  important.  In  the  final  approach, 
the  top-down  and  bottom-up  methods  are  combined.  This  approach  begins  just  like  the 
top-down  approach  with  management  determining  the  strategy  and  tentative  spending 
targets  across  different  categories.  Then,  the  method  moves  to  bottom-up,  where  all 
existing  and  potential  projects  are  ranked  in  a  single,  prioritized  list.  The  breakdown  of 
proposed  spending  is  subsequently  computed  from  this  prioritized  list  of  projects  and 
compared  to  the  tentative  targets  set  by  management.  If  gaps  exist,  management  may 
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reprioritize  active  projects  or  activate  projects  that  are  on  hold  (Cooper  et  a!.,  1998:  83- 
105). 

The  Stage-Gate  Process. 

For  any  of  these  three  goals  (maximizing  value,  achieving  balance,  and  a  strong  link  to 
strategy)  to  be  attained,  a  firm  must  first  have  a  robust  stage-gate  process  in  place.  The 
stage-gate  process  calls  for  management  to  make  difficult  go/kill  decisions  on  projects  at 
different  points  (gates)  in  the  development  process.  The  stage-gate  system  breaks  the 
development  process  up  into  a  pre-determined  discrete  set  of  stages,  each  of  which  is 
preceded  by  a  gate.  Gates  serve  several  purposes  in  the  process.  First,  they  are  quality 
control  checkpoints  where  each  project  is  reviewed  in  detail  to  ensure  high  quality 
execution.  Second,  gates  are  go/kill  decision  points  that  allow  funneling  of  development 
projects — only  the  best  projects  are  allowed  to  proceed  to  the  next  stage.  Finally,  gates 
are  the  points  where  the  game  plan  and  resource  commitments  for  the  next  stage  are 
determined.  One  key  concept  for  an  effective  stage-gate  process  is  that  of  flexibility.  A 
firm  must  be  able  to  tailor  the  process  to  fit  the  size  and  complexity  of  the  project 
(Cooper  et  al.,  1998:  209-212).  Figure  6  shows  how  one  firm  used  three  versions  of  the 
stage-gate  process.  Figure  7  shows  another  way  to  represent  the  stage-gate  process.  A 
funnel  is  used  to  highlight  the  fact  that  the  number  of  projects  in  the  development  process 
decreases  from  one  phase  to  the  next.  Different  portfolio  tools  and  activities  are  also 
shown  on  the  funnel. 
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Figure  6  -  Tailored  Stage-Gate  Process  (Cooper  et  al.,  2002:45) 
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Figure  7  -  The  Development  Funnel  (Cooper  et  al.,  1998;  Wheelwright  and  Clark,  1992) 


Once  a  firm  has  a  stage-gate  process  in  place,  it  must  integrate  its  portfolio 
management  tools  into  that  process.  Cooper  et  al.  (2000)  describe  two  different 
approaches  to  integration.  In  the  first  approach,  the  gates  dominate  the  process. 
Emphasis  is  placed  on  sharpening  the  criteria  used  to  allow  projects  to  proceed  through 
the  gates.  As  long  as  good  project  decisions  are  made  at  the  gates,  the  portfolio  will  take 
care  of  itself.  In  the  second  approach,  a  portfolio  review  dominates  the  process.  One  of 
the  gates  is  replaced  with  a  portfolio  review  where  all  projects  are  rated  against  one 
another.  Go/kill  decisions  are  made  to  ensure  the  portfolio  remains  balanced  and  tied  to 
the  firm’s  strategy.  Here,  the  gates  are  merely  used  as  checkpoints  for  projects  to  ensure 


20 


they  are  on  schedule  and  remain  financially  sound  (Cooper  et  al.,  2000).  The  first 
approach  has  a  project-level  focus,  while  the  second  approach  has  a  portfolio-level  focus. 

Cooper  et  al.  (2000)  continued  their  research  and  discovered  a  heightened  interest  in 
portfolio  management  in  both  the  technical  community  and  the  CEO’s  office.  Their 
survey  of  Industrial  Research  Institute  member  companies  found  that  companies  turned 
to  portfolio  management  for  several  reasons:  to  maximize  return  on  research, 
development,  and  technology  investments;  to  maintain  competitive  advantage;  to  help 
allocate  limited  resources;  to  link  project  selection  with  business  strategy;  to  achieve  the 
right  balance  of  projects;  to  achieve  stronger  focus;  to  improve  horizontal  and  vertical 
communication  within  the  organization;  and  to  provide  more  objectivity  in  project 
selection. 

Cooper  et  al.  (2000)  also  found  that  many  companies  were  having  trouble 
implementing  an  effective  portfolio  management  process.  They  found  four  main  problem 
areas  in  the  application  of  portfolio  management.  First,  most  firms  suffer  from  project 
gridlock — too  many  projects  and  not  enough  resources  to  complete  them.  This  problem 
stems  not  only  from  a  lack  of  resources  but  also  a  lack  of  will  on  management’s  part  to 
scale  down  the  number  of  projects.  Second,  management  does  not  prioritize  projects 
against  one  another.  Projects  are  usually  rated  against  objective  criteria  so  they  all  end  up 
looking  good.  If  projects  are  force-ranked  against  one  another,  then  management  simply 
needs  to  start  allocating  resources  at  the  top  of  the  list  until  the  resources  run  out.  Those 
projects  that  do  not  receive  funding  are  then  put  on  hold  or  are  cancelled.  Third, 
management  makes  too  many  go/kill  decisions  without  solid  project  information.  The 
up-front  homework  on  projects  is  not  done  well  and  management,  in  turn,  makes  difficult 
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investment  decisions  with  poor  quality  infonnation.  Finally,  most  firms  have  too  many 
minor  projects  in  their  portfolio.  A  certain  amount  of  resources  must  be  committed  to 
ambitious  projects  that  promise  breakthroughs  in  technology  or  a  change  in  the  basis  of 
competition.  In  other  words,  these  firms  fail  to  maintain  the  right  balance  of  projects  in 
their  portfolios.  Figure  8  shows  the  effect  that  all  of  these  problems  can  have  on  a  firm’s 
performance  (Cooper  et  al.,  2000). 

Cooper  et  al.  (2004)  served  as  subject  matter  experts  for  the  American  Productivity 
and  Quality  Center’s  study  on  perfonnance  and  best  practices  in  new  product 
development  (NPD).  They  reported  the  results  in  a  series  of  articles,  highlighting  what 
differentiated  the  top  performing  businesses  from  the  rest.  The  study  focused  on  all 
aspects  of  NPD,  including  portfolio  management.  Figure  9  lists  some  portfolio 
management  best  practices  identified  by  Cooper  et  al.  (2004)  and  how  the  businesses 
fared  in  each  area.  The  best  practices  are  listed  in  decreasing  order  of  impact,  with  high- 
value  projects  having  the  strongest  correlation  with  NPD  perfonnance.  The  results 
indicate  that,  despite  an  increased  emphasis  on  portfolio  management,  most  firms  still 
have  a  long  way  to  go  in  terms  of  implementing  these  best  practices  (Cooper  et  al, 
2004). 
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Figure  8  -  Effect  of poor  portfolio  management  on  business  performance  (Cooper  et  al.,  2000) 
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Figure  9  -  Impact  on  performance  of  portfolio  management  best  practices  (Cooper  et  al.,  2004) 

Research  into  Government  Portfolio  Management  Activities 

While  an  extensive  amount  of  research  has  been  completed  in  commercial  new 
product  development  and  portfolio  management,  very  little  has  been  accomplished  in 
assessing  how  portfolio  management  applies  to  government  organizations. 

Brandon  (2004)  suggests  that  there  are  two  barriers  to  implementing  portfolio 
management  in  a  government  organization.  First,  he  states  that  many  government 
organizations  attempt  to  apply  portfolio  management  before  they  even  have  a 
standardized  project  management  process.  It  is  extremely  difficult  to  manage  a  portfolio 
of  projects,  when  each  project  uses  different  tools  or  no  tools  at  all.  Second,  Brandon 
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concludes  that  the  public  sector’s  lack  of  control  of  which  projects  to  implement  hinders 
effective  portfolio  management.  Most  projects  are  dictated  either  by  law  or  by  political 
influence.  Therefore,  portfolio  management  activities  in  a  government  organization  are 
primarily  limited  to  resource  allocation  (Brandon,  2004). 

Greiner  et  al.  (2002)  conducted  an  assessment  of  Air  Force  development  portfolio 
management  practices.  Their  research  consisted  of  a  survey  of  25  respondents  who  were 
involved  in  the  Air  Force  Corporate  Structure  decision-making  process.  The  survey  was 
divided  into  several  sections:  strategic  alignment,  organizational  communication, 
decision  support  tools,  feedback  processes,  risk  and  uncertainty  identification,  weapon 
system  value,  and  the  effects  of  schedule.  Survey  results  were  compared  to  an  ideal 
model  of  commercial  best  practices  derived  from  previous  research.  The  study  found  a 
serious  disconnect  between  Air  Force  portfolio  management  practices  and  commercial 
best  practices.  Table  1  summarizes  the  study’s  findings  (Greiner  et  al.,  2002). 


25 


Table  1  -  Comparison  of  Air  Force  and  Commercial  Best  Practices  (Greiner  et  al.,  2002) 


Success  Factors 

Commercial  Best 
Practices 

Air  Force  Portfolio 
Management  Practices 

Strategic  Focus 

(Cooper  etal..  1998b;  Danila, 

1989;  De  Maio  et  al..  1994;  Griffin, 

1997;  Hall  &  Naudia.  1990;  Lester. 

1998;  Liberatore.  1987;  Souder & 
Mandakovic,  1986;  Weber  et  al.,  1990; 
Wheelwright  &  Clark.  1992) 

Corporate  goals,  objectives,  and 
strategies  must  be  the  basis  for 
portfolio  selection. 

Perception  that  no  process  in 
place  to  measure  project's 
alignment  with  strategic  goals. 

Senior  Management 
Support 

(Calantone  et  al. .  1995;  Cooper  et  al., 
1998b;  Danila,  1989;  Gupta  et  al., 

2000;  Hall  &  Naudia,  1990;  Lester, 

1998;  Liberatore.  1987;  Poolton  & 
Barclay,  1998;  Scott,  2000) 

Senior  management  must  be 
closely  involved  in  R&D  project 
selection  decisions. 

Senior  leadership  support 
perceived  as  a  must  for 
development  project  survival. 

Communication 

(Cooper  et  al.,  1998b;  Danila,  1989; 
Gupta  etal.,  2000;  Poolton  &  Barclay, 
1998;  Souder  &  Mandakovic,  1986) 

Must  establish  good  communi¬ 
cation  links,  both  internal  and 
external. 

Multiple  established  paths  for 
flow  of  strategic  policy 
information. 

Portfolio  Methods 
and  Use 

(Cooper etal.,  1998b;  De  Malo etal., 
1994;  Hall  &  Naudia,  1990;  Weber  et 
al..  1990) 

Multiple  methods  incorporated 
—  used  to  gain  insight  into  each 
project. 

Little  to  no  decision  support 
processes  implemented. 

Primarily  budgetary  focus. 

Flexibility 

(Calantone  et  al.,  1995;  Cooper  et  al.. 
1998b;  De  Maio  et  al.,  1994;  Scott, 
2000;  Weber  et  al.,  (1990); 

Wheelwright  &  Clark.  1 992) 

Able  to  address  resource,  benefit, 
and  outcome  interactions.  Adapt 
to  changes  in  goals,  requirements, 
and  project  characteristics. 

Perceived  lack  of  assessment 
as  to  the  impacts  that  portfolio 
management  decisions  have  on 
other  weapon  systems. 

Decision  Making  at 
Different  Levels 

(Cooper  etal.,  1998b;  Liberatore, 

1987;  Wheelwright  &  Clark,  1992) 

Must  be  able  to  address  the 
organizational  structure  and  the 
decision-making  process  involved 
at  each  level. 

Not  directly  assessed.  Limited 
decision-making  information 
available  and  compressed 
decision-making  time  frame. 

Risk 

(Cooper  et  al. .  1 998b;  Wind  & 

Mahajan,  1988) 

Risk  and  uncertainty  must  be 
addressed  within  the  selection 
process. 

Perception  that  risk  and 
uncertainty  not  adequately 
addressed  during  portfolio 
management  process. 

Multi-functional  Approach 

(Calantone  et  al..  1995;  Griffin,  1 997; 
Gupta  etal.,  2000;  Lester,  1998; 

Scott,  2000) 

Effective  implementation  of 
multi-functional  teams  in  the 
portfolio  management  process. 

Multi-function  evaluation  groups 
at  each  level  of  the  AFCS. 
Perception  of  stovepiping  at 
lower  levels  (Panels/IPTs). 

Customer  Focus 

(Calantone  et  al.,  1995;  Gupta  et  al., 
2000;  Lester,  1998;  Poolton  & 

Barclay,  1998;  Scott,  2000) 

Needs  and  requirements  of 
the  User  assessed  during  the 
project  selection  process. 

User's  validated  need  or 
requirement  key  driver  in 
portfolio  management 
process. 
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Summary 

The  literature  clearly  shows  that  portfolio  management  is  a  key  component  of  a 
successful  new  product  development  process.  Lack  of  an  effective  portfolio  management 
process  can  lead  to  increased  cycle  times.  Top  performing  companies  are  those  that  have 
a  robust  stage-gate  process  in  place.  They  use  portfolio  management  tools  that  enable 
them  to  maximize  the  value  of  their  projects,  maintain  the  right  balance  of  project  types, 
and  ensure  a  strong  link  between  their  projects  and  overall  strategy.  They  focus  most  of 
their  time  and  efforts  on  the  front  end  of  the  development  process,  ensuring  that  they 
select  only  the  best  projects  and  that  they  allocate  resources  properly.  Firms  that  have  an 
effective  portfolio  management  process  in  place  achieve  better  results  on  a  number  of  key 
performance  metrics,  including  faster  cycle  times.  The  lack  of  literature  on  portfolio 
management  in  government  new  product  development  indicates  that  more  research  is 
needed  in  this  area. 
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III.  Methodology 


Research  Goals 

The  primary  goal  of  this  research  was  to  paint  a  clear  picture  of  the  types  of  portfolio 
management  activities  used  by  the  Air  Force.  The  research  attempted  to  answer  several 
questions:  1)  Does  the  Air  Force  use  portfolio  management?,  2)  If  so,  how  do  they  use 
it?,  3)  How  do  the  Air  Force’s  activities  compare  to  commercial  portfolio  management 
best  practices?,  and  4)  What  should  the  Air  Force  do  to  improve  the  effectiveness  of  its 
portfolio  management  activities?  This  research  is  follow-on  to  McNutt  (1998)  and  is  one 
piece  of  the  larger  Cycle  Time  Reduction  Research  Program. 

The  recent  realignment  of  the  Air  Force  Program  Executive  Officer  (PEO)  structure 
provides  a  window  of  opportunity  for  recommendations  from  this  research  to  be 
implemented.  Figure  10  shows  the  PEO  structure  prior  to  1  January  2004.  In  some 
cases,  program  managers  reported  to  a  PEO,  and  in  other  cases,  they  reported  to  a  product 
center  commander.  Figure  1 1  shows  the  current  PEO  structure.  Under  the  old  structure, 
the  dual  reporting  system  made  commanders’  responsibilities  unclear.  It  was  very 
difficult  to  determine  who  the  portfolio  manager  was  for  a  particular  program.  Figure  12 
shows  the  reality  of  the  reporting  process  for  a  typical  acquisition  program  under  the  old 
structure.  The  PEO  realignment  attempted  to  address  this  problem.  Each  product  center 
commander  is  now  dual-hatted  as  a  PEO  in  charge  of  a  particular  portfolio.  A  window  of 
opportunity  exists  because  product  centers  are  still  in  the  process  of  changing  the  way 
they  do  business  to  fit  the  new  system. 


28 


Figure  10  -  Air  Force  PEO  structure  prior  to  1  January  2004  (Looney,  2004) 


Figure  11  -  Current  Air  Force  PEO  structure  (Looney,  2004) 
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Research  Method 


An  in-depth,  inductive,  exploratory  investigation  of  how  an  Air  Force  product  center 
applies  portfolio  management  was  conducted.  Semi-structured  interviews  were 
conducted  with  personnel  in  leadership  and  other  key  positions  at  different  levels 
throughout  the  center.  From  the  interviews,  three  types  of  data  were  collected:  historical 
-  what  they  have  done  in  the  past;  current  -  what  they  are  doing  now;  and  opinion  -  what 
their  plans  are  for  the  future.  Personnel  were  chosen  in  positions  most  likely  to  use 
portfolio  management  or  have  knowledge  of  its  use.  These  positions  included  positions 
in  the  Center  Commander’s  Program  Execution  Group  and  the  commanders  and  deputy 
directors  of  each  wing  and  direct  reporting  group  (DRG).  These  positions  were  chosen 
based  on  the  review  of  commercial  literature  and  on  the  researcher’s  understanding  of 
portfolio  management.  Other  personnel  suggested  by  the  interviewees  were  also 
interviewed.  These  personnel  included  members  of  the  Plans  and  Programs  Division  and 
the  Capability  Planning  Division.  Sample  interview  questions  are  listed  in  the  Appendix. 
Information  Source 

The  Aeronautical  Systems  Center  (ASC)  at  Wright-Patterson  AFB  was  chosen  as  the 
data  source.  ASC  was  selected  for  four  reasons:  1)  it  is  the  largest  product  center  in  the 
Air  Force;  2)  it  is  the  most  essential  to  the  mission  of  the  Air  Force — the  majority  of 
major  weapon  systems  acquisition  in  the  Air  Force  is  done  at  ASC  and  the  ASC 
Commander  is  also  the  PEO  in  charge  of  the  aircraft  portfolio;  3)  it  was  accessible 
because  of  its  proximity  to  the  researcher;  and  4)  its  personnel  were  accommodating  to 
the  researcher.  Personnel  were  interviewed  at  the  Center,  Wing,  and  Direct  Reporting 
Group  (DRG)  levels.  Figure  13  shows  the  current  ASC  organizational  chart.  Figure  14 
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shows  the  personnel  who  were  interviewed  and  where  they  are  located  in  the 
organization. 

Research  Limitations 

As  with  any  research  effort,  this  effort  has  limitations.  It  only  addresses  one  Air  Force 
Product  Center.  It  also  does  not  address  the  corporate  level  Air  Force.  The  reader  should 
keep  these  two  facts  in  mind  as  this  effort’s  findings  and  recommendations  may  not 
directly  apply  to  other  Air  Force  organizations. 


Figure  13  -  Current  ASC  organizational  chart 


32 


Figure  14  -  Personnel  interviewed  for  this  research  effort 


33 


IV.  Results 


Data  was  collected  from  semi-structured  interviews  with  21  personnel  across  ASC. 
This  data  led  to  several  findings  about  the  practice  of  portfolio  management  within  ASC. 
These  findings  are  presented  here  and  the  researcher’s  recommendations  are  presented  in 
the  following  chapter. 

Finding  #1 

Most  people  report  to  be  using  portfolio  management 

The  first  question  asked  during  each  interview  was,  “Do  you  use  portfolio 
management?”  Out  of  the  21  people  interviewed,  71%  (15/21)  said  they  use  it.  Another 
19%  (4/21)  said  they  do  not  use  portfolio  management  because  they  do  not  think  their  job 
requires  it.  The  last  10%  (2/21)  said  they  do  not  use  portfolio  management  because  they 
feel  portfolio  management  is  more  appropriate  for  the  major  commands  (MAJCOMs). 
These  results  are  summarized  in  Figure  15. 


Percent  of  those  interviewed  who  report  to  use 
portfolio  management 


10% 


19°/ 


■  No  (don't  need  it) 

□  No  (not  appropriate) 


Figure  15  -  Percent  of  personnel  reporting  to  use  portfolio  management 
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From  the  number  of  comments  received  from  personnel  interviewed  at  all  levels,  it  is 
also  apparent  that  the  ASC  Commander  has  emphasized  portfolio  management 
throughout  the  center.  One  member  of  the  Commander’s  Program  Execution  Group 
quoted  the  ASC  Commander  as  saying  to  his  wing  commanders,  “Take  a  portfolio 
perspective;  squadrons  are  responsible  for  execution.”  Shortly  following  the  PEO 
realignment,  the  ASC  Commander  established  quarterly  portfolio  reviews  with  the 
purpose  of  looking  at  the  aircraft  portfolio  from  a  capabilities  perspective  instead  of  a 
program  perspective.  Each  portfolio  review  focuses  on  one  or  more  technologies  and 
how  they  affect  each  wing’s  ability  to  provide  a  certain  capability.  At  one  portfolio 
review  attended  by  the  researcher,  the  ASC  Commander  emphasized  the  importance  of 
the  reviews  for  looking  across  capabilities  and  increasing  communication  between 
programs. 

Finding  #2 

At  the  PEO  level,  portfolio  management  activities  focus  almost  entirely  on 
program  execution 

In  order  to  find  out  what  types  of  portfolio  management  activities  are  being  used  in 

ASC,  follow-up  questions  were  asked  of  the  15  people  who  reported  to  be  using  portfolio 

management.  The  following  is  a  summary  of  the  activities  being  conducted: 

Portfolio  reviews  -  Held  quarterly  by  the  PEO  to  review  entire  aircraft  portfolio;  focus 
is  on  how  different  technologies  impact  wings’  abilities  to  provide  capabilities  to  the 
warfighter. 

Execution  reviews  -  Held  weekly  by  the  PEO  to  review  “hot”  program  issues 
identified  by  wing  commanders. 

Capability  Program  Execution  Reviews  -  Semi-annual  portfolio  review  between  PEO, 
Assistant  Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ),  and  MAJCOM. 
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Program  Management  Reviews  -  Quarterly,  semi-annual,  or  annual  program  specific 
review  held  at  the  wing  or  DRG  level. 

Executive  Weapon  System  Reviews  -  Portfolio  review  held  at  the  wing  or  DRG  level. 

Acquisition  Strategy  Panels  -  Review  of  a  specific  program’s  acquisition  strategy  (or 
plan  of  action)  by  all  major  program  stakeholders. 

Financial  reviews  -  Wing  or  DRG  level  review  of  year-to-date  status  of  financial 
execution. 

Requirements  and  Planning  Council  (RPC)  -  Forum  between  wing  and  MAJCOM 
used  for  prioritization  and  selection  of  program  initiatives  and  new  requirements. 

Program  Prioritization  Process  (PPP)  -  Wing  level  process  used  to  prioritize  work  and 
make  rational  resource  allocation  decisions. 

All  but  one  of  these  activities  focus  only  on  the  execution  of  existing  programs.  Only  the 
RPC  gives  any  consideration  to  the  front  end  of  the  development  process  (i.e.,  project 
selection). 

Finding  #3 

There  is  very  little  front-end  portfolio  management  work  being  conducted  at  the 
PEO  level 

During  this  research  effort,  no  tools  were  identified  at  the  PEO,  wing,  or  DRG  levels 
that  support  the  front  end  of  the  development  process.  Two  of  the  people  interviewed  felt 
that  portfolio  management  consists  mainly  of  project  selection  activities.  The  same  two 
people  also  felt  that  project  selection  activities  were  more  appropriate  for  the  MAJCOMs. 
However,  some  project  selection  work  is  accomplished  at  the  wing  level.  One  wing 
conducts  semi-annual  RPCs  for  each  platform.  During  an  RPC,  the  wing  works  with  the 
MAJCOM  to  prioritize  and  select  program  initiatives  and  new  requirements.  Figure  16 
shows  how  the  Air  Force  acquisition  process  and  ASC’s  portfolio  management  activities 
compare  to  the  commercial  development  funnel  (Figure  7). 
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Capability  Program  Execution  Review 
Quarterly  portfolio  reviews 
Weekl  y  executi  on  revi  ews 
Program  Management  Reviews 
Executive  Wteapon  System  Reviews 
Financial  reviews 
Acquisition  Strategy  Panels 
Program  Prioritization  Process 


Requirements  and  Flaming  Councils 


Figure  16  -  Air  Force  acquisition  process  and  ASC portfolio  management  activities 


The  Air  Force  acquisition  process  is  more  of  a  development  tunnel  than  a  funnel.  This  is 
due  to  the  fact  that  there  are  no  mechanisms  in  place  to  terminate  a  program.  Also,  while 
the  majority  of  activities  in  the  commercial  funnel  support  the  front  end  of  the  process, 
the  majority  of  activities  in  ASC  support  the  back  end  of  the  process  (program 
execution).  Only  one  of  the  center’s  portfolio  management  activities  supports  any  front- 
end  work. 
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Finding  #4 

There  are  few  effective  tools  available  to  support  portfolio  management 

The  System  Metric  and  Reporting  Tool  (SMART)  was  the  only  tool  found  in  ASC  to 

support  portfolio  management.  SMART  is  a  common  reporting  and  management  tool 

developed  by  SAF/AQ  and  Air  Force  Materiel  Command  (AFMC).  According  to  the 

SMART  Concept  of  Operations  (CONOPS), 

SMART  is  intended  to  provide  program  managers  a  daily  management  tool  with 
authoritative  infonnation  and  automated  reporting  to  the  program  execution  chain  and 
to  provide  senior  acquisition  executives  and  their  staffs  reliable  insight  into  program 
and  portfolio  health.  (ESC,  2004) 

All  acquisition  category  (ACAT)  I,  II,  and  III  programs  are  listed  in  SMART.  PEOs  also 
have  the  ability  to  designate  other  efforts  within  their  portfolios  as  programs  in  SMART. 
Every  program  in  SMART  is  assigned  to  a  portfolio,  under  the  authority  of  a  PEO. 
Through  the  use  of  common  business  rules  and  metrics,  SMART  consolidates  individual 
program  health  and  status  infonnation  into  an  overall  rating  for  each  portfolio  (ESC, 
2004).  Figure  17  shows  an  example  of  a  portfolio  summary  screen. 


38 


MART 


V 


Aircraft 


System  Metric  &  Reporting  Tool 


Portfolio  Summary 

•  Action  Officers 

•  Program  Analyzer 


0 


Overall  Health 


|  G  |  Cost 
|  G  |  Schedule 
|  G  |  Performance 
a  Funding 

Management 
G  User  Satisfaction 


r 


Portfolios  )  (  Programs  )  (  “Q  SP0  support  ) 


Portfolio  Summary 


Help?) 


PEO/AC  :  Aircraft 

Lt  Gen  Looney  III,  William  R 

Program  Executive  Officer 
(937)  255-5714 
785-5714 


Introduction: 

ASC  is  constantly  evolving  to  meet  the  warfighter’s  needs  tomorrow— our  success  is  the 
warfighter's  success.  We're  focusing  on  speed,  agility,  and  credibility,  to  rapidly  deliver 
war-winning  capabilities  to  the  men  and  women  on  the  tip  of  the  spear  who  need  them. 
SMART  is  a  powerful  tool  that  helps  us  make  that  vision  a  reality. 

Mission: 

Rapidly  develop,  acquire,  modernize  and  sustain  the  world's  best  aerospace  systems. 

Vision: 

Birthplace,  heme  and  future  of  aerospace.  Rapidly  delivering  war-winning  capability 
with  innovation,  trust  and  warrior  spirit. 

Short  Description: 


Description: 


Last  Modified  on  22  Jan  04  at  17  :54  by  Mr.  Giroux.  Brian  E  . 


History | 


Figure  17  -  Portfolio  summary’  screen  in  SMART 


The  use  of  SMART  has  been  mandated  by  SAF/AQ  for  all  monthly  acquisition  reporting. 
SMART  is  widely  used  across  ASC,  but  it  only  supports  program  execution  activities.  It 
is  used  simply  as  a  reporting  tool  for  existing  programs.  The  researcher  was  not  able  to 
identify  any  capabilities  within  SMART  that  support  project  selection. 

Finding  #5 

There  is  no  clear  consensus  in  ASC  on  what  defines  a  program  and  what  defines  a 
project 

Officially,  there  are  70  programs  listed  in  the  aircraft  portfolio.  Some  of  those 
programs  are  entire  platforms  (like  the  B-2).  Some  platforms  have  multiple  programs  in 
the  portfolio  (like  the  B-52).  Figure  18  shows  the  complete  listing  of  programs  in  the 
aircraft  portfolio. 
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Special  Ops 

AC-130U  Gunship 
TSSAM  Litigation 

CSAR  Personnel  Recovery  Vehicle 
CV-22 

Reconnaissance 

Compass  Bright 
Compass  Call 
Senior  Scout 
Senior  Year  (U -2  ; 

Airborne  Signals  Intel  Program 

M aritim  e  P  atrol  Aincraf 

Sensor  Replacement 

Theater  Airborne  Recon  System 

USAEDS 

Commando  Solo 

Imaging  &  Targeting  System 

NCCT 

Predator  A 

PredatorB 

RC-135 

Global  HavA 


Fiqhter  Attack 

A-10  Frecisior  Engagement 

pnvg 

F-15 

F-ie 

JcirtHelmet-r.fcurteS  Cueing  System 
LITENING  II  PrcPucticn 
SNIPER  XR  ATP 
F-15  Aircre.'.  Training 
F-15  Aircrew  Traininq 

Lonq  Ranqe  Strike 

CLCVefS 

B-1B 

F-117 

B-2 

B-52  Avionics  Mb  life  Improvement 

B-52  CONNECT 

B-52  SA  Defensive  I  me  reverent 

Aqile Combat  Support 

Aqinq  Airoraft  F-agram 
PRAM 

Aergmeoicsl  Systems 
Life  Sucocrt  Svetms 
DMO 

Engine  CIF 
F110SLEF 
PLAID 

TEWS  ID  EDM' 


T  raininq 

USAFA  dice  Reolacemert 
T-1A 

T-5A  JPATS 

T-38  Pro  puls  icn  Moo  Prgm 
T-38  Avionics  Uegreae  Program 
T-38  Escace  S/stem  UooraOe 

Mobility 

C-32A  Communications  Uograoe 
C-27 

C-AJ  Lese 
C-OB  VIPSAM 
LAI  PCM 
C-5MF 
C-E  RERP 
KC-767 
Halvorsen 
Tunre' 

C-130  AfvP 

C-130j 

C-17 

KD-10  GATM 
KC-135  GATM 
C-E  Aircres  Twining 
C-130  Aircrew  Training 
FC-10  Aircre.'.  Trairina 
KC-135  Aircre-v  Training 


Figure  18  -  Official  list  of  programs  in  the  aircraft  portfolio  (Looney,  2004) 


The  researcher  found  no  documented  rationale  for  why  those  70  programs  were  chosen. 
None  of  the  personnel  interviewed  could  state  for  certain  how  or  why  those  programs 
were  chosen.  Several  people  thought  the  programs  were  most  likely  based  on  program 
size,  dollar  value,  level  of  oversight,  or  interest  from  higher  headquarters.  Also,  several 
personnel  interviewed  at  the  wing  level  did  not  agree  with  the  official  program  list.  For 
example,  one  wing  claims  to  have  well  over  100  different  programs  but  less  than  10  are 
on  the  PEO’s  list.  Another  wing  claims  to  have  just  one  program.  Other  wings  claim  to 
have  a  small  number  of  programs,  all  of  which  go  to  the  PEO  level.  Clearly,  there  is  no 
consensus  across  the  center  as  to  what  level  of  effort  constitutes  a  program  versus  a 
project. 
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Finding  #6 

Projects  within  ASC  are  not  tracked 

The  PEO  has  no  visibility  beyond  the  70  programs  listed  in  his  portfolio.  There  are 
likely  to  be  tens,  if  not  hundreds,  of  more  projects  ongoing  in  the  center  than  programs. 
Figure  19  shows  one  example  program,  the  F-15,  which  has  20  sub-projects  within  it. 


F-15Piojects  i 

220 E  Engines 

Avionics  Control  Unit  | 

ALE-58  Countermeasures  Dispenser 

APG-63M1  Radar 

Mode  5  IFF 

Digital  Video  Recorder 

Advanced  Display  Core  Processor 

Embedded  GPS/INS 

Air  Data  Processor 

IFF  /  AAI 

ALQ-135  Band  1.5 

JHMCS  Integration 

Programmable  Armament  Control  System 

VHF  Radios 

AESA  Radar  (E) 

OFP  Suite  5 

TEWS  Intermediate  Support  SystemTlP 

OFP  Suite  6 

RWR  Upgrade 

Figure  19  -  Current  projects  within  the  F-15  program 


The  PEO  has  no  apparent  visibility  into  these  projects.  Assuming  there  are  other 
programs  in  the  portfolio  similar  to  the  F-15,  then  the  PEO  lacks  visibility  into  the  vast 
majority  of  work  going  on  in  his  portfolio.  Since  there  are  no  clear  program  or  project 
criteria,  there  may  be  multi -million  dollar  projects  ongoing  in  the  center.  Since  projects 
are  not  tracked,  the  PEO  lacks  visibility  and  control  over  most  of  his  portfolio. 

Finding  #7 

ASC  has  not  taken  a  leading  role  with  respect  to  the  most  essential  part  of 
portfolio  management 

The  ability  to  control  and  limit  the  number  of  development  projects  in  a  portfolio  is 
the  most  essential  concept  in  commercial  portfolio  management.  In  ASC,  the  PEO  has 
little  to  no  control  over  what  he  is  asked  to  do  or  when  is  he  asked  to  do  it.  If  a 
MAJCOM  has  a  requirement  for  a  system  and  funding  to  support  it,  then  the  PEO  must 
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execute  that  program.  He  has  almost  no  ability  to  control  what  programs  or  projects 
come  into  his  portfolio.  Project  selection  activities  have  essentially  been  left  to  the 
MAJCOMs  and  to  the  corporate  Air  Force.  The  PEO  lacks  visibility  at  the  project  level 
within  the  center  and  has  few  control  mechanisms  to  manage  new  projects  within  existing 
programs.  The  PEO  is  unable  to  accomplish  proper  resource  allocation.  The  PEO  is 
limited  to  minor  reprogramming  actions  and  shifts  within  programs.  The  PEO  also  does 
not  appear  to  have  the  authority  or  ability  to  terminate  programs,  which  is  a  key  concept 
in  commercial  portfolio  management.  The  PEO  is  primarily  charged  with  overseeing 
program  execution  and  allowing  programs  to  pass  to  the  next  phase  of  the  acquisition 
process. 
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V.  Recommendations 


The  research  findings  lead  to  several  recommendations  that  will  serve  as  a  first  step 
toward  improving  the  effectiveness  of  portfolio  management  activities  within  ASC.  The 
recommendations  are  based  on  commercial  portfolio  management  best  practices,  as 
described  in  the  literature  review. 

Recommendation  #1 

Focus  more  on  the  front  end  of  the  development  process 

The  front  end  of  the  commercial  development  funnel  is  the  most  important 
component.  Cooper  and  Kleinschmidt  (1988)  found  that  successful  projects  spent  twice 
as  much  money  and  twice  as  many  man-days  on  front-end  work  than  projects  that  failed. 
Wirthlin  (2000)  found  a  strong  link  between  budget  instability  and  poor  front-end  work. 
As  shown  by  Figure  20,  approximately  80%  of  a  system’s  total  life  cycle  costs  are 
determined  up  front  and  early  in  the  acquisition  process. 


—  —  -  Percent  Costs  Locked  In  by  Early  Decisions 
■■  —  Percent  Costs  Actually  Expended 
Cost  Reduction  Opportunities 


Figure  20  -  Distribution  of  costs  over  typical  program  lifecycle  (Butler,  2004) 
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Also,  if  the  Air  Force  expects  a  PEO  to  manage  a  portfolio,  then  that  PEO  must  have 
control  over  what  comes  into  his  portfolio.  The  new  3-star  PEOs,  resulting  from  the 
recent  PEO  realignment,  should  have  more  control  and  influence  over  the  project 
selection  process  than  their  predecessors.  A  key  focus  of  successful  corporate  CEOs  is 
where  their  business  is  going  (strategic  direction)  and  how  it  is  going  to  get  there  (project 
selection  strategies).  This  should  also  be  the  focus  of  Air  Force  PEOs.  To  help  establish 
this  focus,  the  Requirements  and  Planning  Council  concept  should  be  adopted  across 
ASC  to  foster  communication  between  the  wings  and  the  MAJCOMs  and  provide  ASC 
input  into  the  selection  of  new  projects. 

Recommendation  #2 

PEO  must  establish  and  document  clear  criteria  that  define  a  program 

Without  these  criteria,  existing  programs  cannot  be  effectively  prioritized.  Effective 
program  prioritization  is  necessary  for  proper  resource  allocation.  Most  personnel 
interviewed  stated  that  they  had  been  tasked  at  one  time  or  another  to  prioritize  their 
programs  for  the  PEO.  Almost  all  of  them  said  that  this  was  impossible  for  them  to  do. 
There  are  no  criteria  or  tools  in  place  to  prioritize  programs  with  widely  varying  levels  of 
effort  (i.e.,  the  B-2  compared  to  a  B-52  avionics  upgrade).  Along  with  clear  program 
criteria,  there  must  be  clear  project  criteria  to  distinguish  between  the  two. 
Recommendation  #3 

Establish  control/insight/visibility  into  the  start  of  projects  at  the  center,  wing,  or 

DRG  level 

With  the  majority  of  work  being  done  at  the  project  level,  the  PEO  must  have 
visibility  at  the  project  level  in  order  to  effectively  manage  his  portfolio.  There  must  be  a 
data  collection  method  at  the  project  level  to  provide  the  appropriate  level  of  insight  and 
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visibility.  An  annual  project  level  data  call  is  one  possible  way  to  accomplish  this. 

Along  with  the  data  call,  the  PEO  must  establish  clear  criteria  that  define  a  project  and 
the  requirements  for  initiating  a  project. 

Recommendation  #4 

Develop  a  way  to  quantify  the  value  of  military  capability 

Maximizing  value  in  terms  of  a  particular  business  objective  is  one  of  the  three  main 
goals  of  commercial  portfolio  management.  Value  is  fairly  easy  to  measure  in  the 
commercial  world.  It  is  most  often  measured  with  dollars.  Therefore,  the  goal  of  a 
firm’s  portfolio  management  process  might  be  to  maximize  profit.  In  the  DoD,  value  is 
much  more  difficult  to  measure.  To  a  warfighter,  military  capability  is  valuable. 
Currently,  there  is  no  method  in  place  to  quantitatively  measure  the  capability  provided 
by  a  particular  military  system.  In  order  to  effectively  prioritize  programs  and  to  assist 
with  the  selection  of  new  programs,  the  military  must  establish  a  way  of  quantifying  the 
value  of  military  capability. 

Cost  of  Delay  Analysis  (CoDA)  is  one  tool  identified  by  the  researcher  that  may 
support  this  process.  CoDA  was  first  developed  in  the  commercial  world  in  1983  by  Don 
Reinersten.  It  was  adapted  for  use  on  military  projects  in  1998  by  SAF/AQ,  with  the 
assistance  of  Mr.  Reinersten.  CoDA  is  a  tool  that  quantifies  the  value  of  cycle  time  on 
specific  projects  and  creates  tactical  decision  rules  to  assist  with  making  trade-offs 
between  system  cost,  schedule,  and  performance.  For  military  projects,  CoDA  assigns  a 
dollar  amount  to  the  value  of  a  system  by  subtracting  the  cost  of  the  system  from  the 
estimated  benefits  of  the  system.  Benefits  are  typically  dollarized  by  multiplying  the 
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costs  of  existing  systems  by  an  improvement  ratio  that  accounts  for  the  additional 
capability  of  the  new  system  (Butler,  2004). 

Recommendation  #5 

Further  investigate  the  benefits  of  the  capability  manager  concept 

The  capability  manager  (CM)  concept  is  currently  being  tested  at  ASC.  A  CM  is  a 
single  person  responsible  for  delivering  an  entire  capability  to  the  warfighter.  Figure  21 
shows  some  of  the  top  level  responsibilities  of  a  CM.  A  CM  would  manage  cross-cutting 
capabilities,  like  Airborne  Electronic  Attack  and  Data  Links,  which  touch  a  number  of 
systems  and  organizations.  A  CM  would  provide  a  single  face  to  the  user  for  a  given 
capability,  rather  than  multiple  faces  that  each  provides  only  a  portion  of  a  capability.  A 
CM  would  have  control  over  funding  for  the  entire  capability  so  that  he  or  she  could 
make  effective  and  appropriate  resource  allocation  decisions. 


Capability  Manager  (CM) 

Network  Enabled  Warfare 


•  Transforms  war-fighting  needs  into  recommended  program 
strategies  for  systems  and  products 

■  Supports  the  war-fighters  in  defining  their  capability  needs 

■  Generates  alternative  system  (or  system  of  systems)  concepts 

■  Conducts  analyses  to  describe  military  utility,  enabling 
technologies,  operations  and  support  factors,  technical,  cost, 
schedule,  and  operational  risks,  life-cycle  cost  of  ownership,  and 
other  aspects  of  potential  solutions 

■  Recommend  preferred  solutions  based  upon  analysis  results 

■  Develops  executable  acquisition  strategies  for  systems  and 
products 

■  The  results  of  the  development  planning  process  supports 
decision-makers  within  the  war-fighting  and  acquisition 
communities 


Quality  Information  for  Quality  Decisions  to  MAJCOMs 


Source:  ASC/XR  Imptementaticn  Flan  &  ASC  Transiticn  CONOPSfcr  Acquisition  VUrgs/G)-oups/Squa<±cns  Ogsnization 

Integrity  -  Service  -  Excellence 


Figure  21  -  Capability  manager  responsibilities  (Urschel,  2004) 
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As  the  Air  Force  moves  to  a  capabilities-based  acquisition  system,  a  CM  may  serve  as 
a  more  appropriate  portfolio  manager  than  the  current  PEOs.  According  to  many  of  the 
personnel  interviewed,  the  CM  concept  has  buy-in  from  the  ASC  Commander  and  the 
Chief  of  Staff  of  the  Air  Force.  However,  many  people  within  the  wings  and  DRGs  in 
ASC  do  not  understand  the  benefits  of  the  CM  concept.  ASC  should  identify  a  successful 
pilot  program  to  demonstrate  the  benefits  of  the  concept  to  the  workforce. 

Conclusion 

Portfolio  management  has  been  an  effective  cycle  time  reduction  tool  in  the 
commercial  world  for  several  decades.  Long  cycle  times  have  been  a  problem  in  the 
DoD  for  many  years  and  continue  to  be  a  problem  today.  Previous  research  identified 
portfolio  management  as  an  essential  component  of  an  overall  cycle  time  reduction  plan. 

This  research  effort  has  shown  that  one  product  center  within  the  Air  Force  claims  to 
be  using  portfolio  management.  However,  data  gathered  from  the  interviews  suggests 
that  this  product  center’s  portfolio  management  activities  focus  mainly  on  program 
execution  at  the  back  end  of  the  development  process.  In  comparison,  the  activities  of 
successful  commercial  firms  focus  mainly  on  project  selection  at  the  front  end  of  the 
development  process.  A  shift  in  focus  to  the  front  end  of  the  development  process,  along 
with  other  recommendations,  will  ensure  that  increasing  cycle  times  do  not  continue  to 
plague  the  Air  Force  acquisition  process. 
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Recommendations  for  Further  Study 

To  further  benefit  the  Air  Force,  this  research  effort  should  be  continued.  The  use  of 
portfolio  management  by  other  Air  Force  product  centers  and  by  the  corporate  level  Air 
Force  should  be  investigated.  The  SMART  tool  should  be  assessed  for  its  usefulness  as  a 
portfolio  management  tool.  Finally,  further  studies  on  the  implementation  of  the 
capability  manager  (CM)  concept  should  be  conducted. 
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Appendix:  Sample  Interview  Questions 


1 .  Does  your  organization  use  portfolio  management? 

2.  How  does  your  organization  define  an  acquisition  program? 

3.  How  often  do  you  have  program/portfolio  reviews? 

4.  What  information  is  covered  at  these  reviews? 

5.  Who  determines  what  information  is  covered? 

6.  Is  there  a  standard  format  for  displaying  program  infonnation? 

7.  Are  there  any  specific  tools  used  to  display  information  (graphs,  tables,  maps)? 

8.  Are  there  different  reviews  for  programs  in  different  acquisition  categories? 

9.  Are  reviews  used  to  rank/prioritize  projects  or  are  they  simply  used  to  provide  status 
updates  to  senior  leaders? 

10.  Are  projects  ever  tenninated  at  these  reviews?  If  not,  is  there  a  process  or  screen  in 
place  to  terminate  poorly  perfonning  projects? 

11.  Are  projects  ever  prioritized  across  the  portfolio  as  a  whole  or  just  within  a  given 
area? 

12.  How  are  resources  allocated  to  various  projects  in  your  organization? 
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